Technique For Online Video-Gaming With Sports Equipment

ABSTRACT

Methods and apparatus for use in an Internet-of-Things (IoT) system are presented. A method performed by a network node in an Internet-of-Things (IoT) system is presenting in an example embodiment. According to this example method, a network node or a plurality of network nodes working collaboratively can obtain data captured by one or more IoT devices and associated with an interactive activity with which different participants interface at different sites. Furthermore, the network node(s) may determine a state of the activity by applying the data to a data model. Moreover, the network node(s) can, in an example method, provide one or more feedback signals to feedback devices associated with one or more of the different sites to actualize the determined state of the activity, where at least two of the different sites have disparate, meaning non-identical, feedback device sets.

TECHNICAL FIELD

The application relates to systems, methods, and apparatus forparticipating in online gameplay with other participants in differentlocations.

BACKGROUND

Internet-of-Things (IoT) is the vision of virtually all objects beingconnected to the internet, where the objects can be anything from simplesensors to large, sophisticated machinery. For instance, in the physicalfitness and sports space, wearable IoT devices have been designed totrack athletes' training regimens and to optimize the fitness plans ofeven the amateur athlete. Contrary to traditional physical activities,such as organized team sports and cross training, existing IoT devicestailored for use in video games can only be played at home in front of atelevision with expensive video game console devices. As a result, thesedevices rarely incite players to engage in meaningful physical activityin order to lead a healthy life.

Prior art can be found in, e.g.: US 2006/0148594 A1, which generallyrelates to a smart communicating sports equipment; US 2009/0029754 A1,which generally relates to tracking and interactive simulation of realsports equipment; WO 2007/006083 A1, which generally relates to a gameof chance for playing in conjunction with a sporting contest such asRugby Union and Rugby League, in which a ball is used and in whichscoring areas in the sporting contest are divided into a number ofscoring zones with the results of the game of chance being determined bythe position or positions of the ball or a movement or movements of theball into the scoring zones; U.S. Pat. No. 8,834,303 B2, which generallyrelates to an arena baseball game system and method for playing anelectronic interactive spectator participation game; U.S. Pat. No.7,789,742 B1, which generally relates to a system that wirelesslyintegrates actual golf equipment with the computer and the internet toallow players remotely located from one another to play a competitivesimulated game of golf; US 2009/0023522 A1, which generally relates to amethod of managing the real-time play performance and play flow betweengeographically remote players of a golf type game comprising real golfshots where a hit ball traverses and encounters a real physical surface;and US 2008/0242409 A1, which generally relates to video feedsynchronization in an interactive environment.

Data from the smart sports equipment and, for example, squash courts aregenerally processed locally, whereby high-level information, for examplea score of a game, is shared across sites and players.

In addition, IoT devices have been implemented as smart sports equipmentand in smart sport spaces (e.g., gyms, courts, etc.), but up to now thistechnology has been utilized exclusively for physically playingtraditional sports. For at least these reasons, improved techniques areneeded for utilizing IoT devices and systems to improve the physicalhealth of individuals and maximize the utility of available smartathletic devices and spaces.

SUMMARY

A network node in an IoT system and a method executed by a network nodein an IoT system according to the present disclosure are outlined in theindependent claims. Variants of the network node and the method executedby the network node are set out in the dependent claims. We furtherdescribe a computer program comprising instructions which, when executedby at least one processor of a network node, case the network node tocarry out the method according to variants as described herein. Furtherstill, we describe a carrier containing such a computer program.

One or more embodiments herein allow for improved utilization of IoTdevices and systems in competitive or cooperative sports, games, orother activities whereby one or more participants may interface with theactivity via an electronic device (e.g., a video game console and/ortelevision), which other participants may interface in a traditionalphysical manner that involves a lesser degree of virtual interactionwith an activity space.

Some embodiments, for example, include a method executed by a networknode (or a plurality of network nodes working in concert) in anInternet-of-Things (IoT) system that includes obtaining data captured byone or more IoT devices and associated with an interactive activity withwhich different participants interface at different sites. In addition,the network node (or nodes) may determine a state of the activity byapplying the obtained data to a data model. Furthermore, the networknode(s) may provide one or more feedback signals to feedback devicesassociated with one or more of the different sites to actualize thedetermined state of the activity. What is more, in some examples, atleast two of the different sites may have disparate (i.e.,non-identical) feedback device sets.

Further embodiments include corresponding apparatus, computer programs,and computer program products.

BRIEF DESCRIPTION OF THE DRAWINGS

These and other aspects of the present disclosure will now be furtherdescribed, by way of example only, with reference to the accompanyingfigures, and in which:

FIG. 1 illustrates an IoT system corresponding to example embodiments ofthe present invention;

FIG. 2 illustrates a method performed by a network node (or collectivenetwork nodes) according to one or more embodiments;

FIG. 3 illustrates further aspects of a method performed by a networknode (or collective network nodes) according to one or more embodiments;

FIG. 4 illustrates aspects of an example network node in exampleembodiments of the present invention;

FIG. 5 illustrates aspects of an example IoT device in exampleembodiments of the present invention.

FIG. 6 illustrates an example process chain according to embodiments ofthe present invention.

FIG. 7 illustrates an example implementation according to embodiments ofthe present invention.

FIG. 8 illustrates communication links between elements according toexample embodiments of the present invention.

DETAILED DESCRIPTION

The present disclosure describes example techniques for utilizingnetwork nodes in a cloud-based, IoT-capable network to construct,maintain, and render a state of a multi- or solo-participant activity toone or more participant-specific activity environments/feedback devicesets. This allows participants in disparate locations and withpotentially mutually-exclusive sets of available feedback devices tocontemporaneously participate in the activity, whether competitively orcooperatively, by interfacing/interacting with the rendered activitystate via data input to the network node(s) and feedback implementingthe rendered state from the network node(s) to one or more feedbackdevices in the participant feedback device set. This input data caninclude one or both of observed data generated by IoT devices withoutsignificant participant interaction (e.g., environment sensors,microphones, cameras) or via direct input to the activity state viaphysical movement or electronic signals (e.g., a video game controlleror “smart” sports/activity apparatus). In some examples, the feedbackprovided to the participant by the network node(s) may beparticipant-specific and can depend upon the participant's particularenvironment/feedback device set and the generated activity statecalculated by the network node(s).

This flexible, device-set- and participant-specific activity staterendering functionality, does not limit participants to those with apreferred activity style or prescribed/concrete set of available IoTand/or feedback devices. Instead, some participants may physicallyperform the activity (e.g., run and swing at a ball with a racquet),whereas another participant may use a computer and input/output (I/O)device (e.g. a virtual racquet, video game controller, laptop, tablet,etc.) to participate in the activity virtually (e.g., by interactingwith the activity, the activity space, and other participants through anelectronic device). In some examples, the participants may interfacewith the interactive activity in a feedback-device-rendered variation ofa traditional real-life form of the activity.

In some aspects of the present disclosure, the activity may comprise acompetition between or among a plurality of the different participantsor a competition between a single participant against a benchmark orprevious score.

In an aspect of the present disclosure, these and other features may berealized by an activity-controlling network node or network nodes that,in some examples, form a cloud computing environment. The networknode(s) can receive observed data from different IoT devices configuredto capture certain aspects of the activity (e.g., images or video viaone or more IoT cameras, temperature via thermometers, sounds viamicrophones, or any other type of sensor that can provide the networknode(s) useful data regarding a present and/or future state of theactivity).

Based on this data provided by the IoT devices located at one or moreactivity sites (e.g., real-world, virtually rendered, or mixed activityspaces), one or more data processing modules in the network node(s) canprocess the data according to a static (i.e., predefined) or dynamic(i.e., “smart” or specific to IoT device-provided data, malleable andenvironment/state dependent) data model. Based on the application of theIoT-device-provided data to the data model, the network node(s) canconstruct a current state of the activity. This state may include one ormore conditions, parameter values, participant or object (e.g., ball,Frisbee, racquet) locations or movement characteristics, site-specificconditions, network profile, available device and/or network resources,time of day, history, etc. that defines a current (or recently observed)state of the activity or game.

Based on this current state (and/or past states), the network node(s)can provide feedback to some or all of the participants to render areal-time (or quasi-real-time) dynamic activity experience for theindividual participants via their unique set of feedback devices. Thisfeedback, for instance, may include rendering an interactive virtualactivity space to an online video game user (e.g., displaying a virtualactivity space on a television and providing vibratory tactile feedbackvia a video game controller) and/or a real-world rendering (e.g., viaimage projection on one or more walls, sounds, tactile feedback) viaactivity-, state-, and/or participant-specific feedback instruments(e.g., vibrating squash racquet, lights on a dark basketball court,speakers), thereby providing a representation of the network-maintainedactivity state to participants more physically participating in theactivity. By maintaining the same present activity state (and optionallypredicting one or more future states), the network node(s) of an IoTnetwork environment allow for interactive, real-time activityparticipation for participants who are located anywhere that has accessto the network and whose method of interacting with the activity andother participants can vary significantly from almost purely physical(i.e., traditional or “real-world”) to almost purely virtual (e.g., viaa virtual reality headset, video game controller, and otheractivity-specific sensory immersion tools).

A myriad of activities can be optionally presented for participantinteraction by the present embodiments, from single-player chess tovirtual multi-participant golf matches, and from traditional squash(with projected statistics about score, performance, etc.) and any othersingle-, and/or multiplayer video games that are traditionally playedthrough home video game consoles or computers via a TV screencommunicating with one or more network nodes over wired and/or wirelesscontent provider networks (e.g., 4G wireless networks, the Internet).For example, one player alone in a squash court in Budapest could hitthe squash ball to the front-wall, while another player alone in asquash court in Stockholm could hit a squash ball to the front-wall, andaccording to the locations the two balls hit the wall, a projected“brick breaker game” (computed in a network node in a California datacenter, and projected in the two courts the same way in effectively realtime) could be destructed simultaneously. The proposed systems allow formulti-purpose usage of widely available and relatively inexpensiveIoT-based sensors and/or feedback devices located near a participant(e.g., mounted on a squash court and on smart sports equipment). The I/Ointerfaces can be implemented by the aforementioned equipment, which isoften already necessary and in place in a particular activity location,in order to enrich a traditional sporting (activity) experience. Thecomputations related to the actual activities, sports, competitions,games can be performed in the cloud, as opposed to local (e.g., activityspace specific) computation performed in game consoles or computers, andthe network connectivity of the activity spaces, already in place, canbe used to transfer the sensor data to the network nodes comprising thecloud network, which uses the data to monitor a state of the activityand render the state to each participant via a particular user interfaceand device-set-specific feedback to the activity spaces.

FIG. 1 illustrates an example IoT communication system 100 (“IoTsystem,” “system”) that includes IoT devices 102 in communication withone or more network nodes 106, which together may operate as a cloudcomputing network. In such a cloud computing network configuration, thenetwork nodes 102 may operate collaboratively, as each network node mayutilize its own processing resources, memory, modules, etc. inconjunction with those of other network nodes to perform aspects of thepresent disclosure. This is illustrated in FIG. 1 by the stacked networknodes 106, indicating that although a single network node 106 mayinclude each of the components and perform all of the aspects of thepresent embodiments attributed to a network node 106, in some examples aplurality of network nodes 106 may work in conjunction to do so.Therefore, in some embodiments, a separate network node 106 may performone aspect of an embodiment and may contain an associated module fordoing so, and different network node 106 may perform a separate aspectof the embodiment and may contain the associated module for performingthat second aspect. As such, the term “network node” can be substitutedfor the term “network nodes” as used herein.

IoT devices 102 are configured to generate data 103 and transmit thatdata 103 to the network node 106. The data 103 may include any type ofdata, including data that the particular IoT device 102 is configured togenerate, transmit, and/or otherwise output. For instance, the IoTdevice 102 may be a sensor, such as, but not limited to, a temperaturesensor/thermostat, a camera, microphone, or any other device configuredto communicate autonomously with one or more other wireless devices ornetwork nodes. In some examples, the IoT devices 102 may be configuredto generate data associated with a particular activity, such as, but notlimited to a sport, video game, athletic contest, or any otherobservable activity in which one or more participants can take part inthe real world or virtually. In the present disclosure, an exampleembodiment will be presented wherein one or more participants compete ina game of squash (racquet sport) that is observed by one or more IoTdevices 102 and rendered by network nodes 106. This serves as an exampleuse-case that illustrates the ways in which different participants canparticipate in a fast-paced collaborative or competitive activity viasophisticated functional operation and state rendering on the networkside. Though this is one example use-case, it is not meant to beregarded in any way as limiting.

As shown in FIG. 1, the system 100 can provide a hub for activityparticipation at disparate locations, or “sites,” such as site A andsite X. Each site may include one or more IoT devices 102, as well asone or more feedback devices 112. These devices 102, 112 may includesets of identical, completely different, or partially-overlapping setsof IoT and feedback device types. As shown, for example, site A includesIoT devices 102A and 102B, whereas site X includes IoT devices 102X and102Y. Likewise, the two sites contain feedback devices 112A and 112B,and 112 x and 112Y, respectively. In some examples, an optional(indicated by the dashed outline) data collector device or module108A/108X may collect data 103 observed by IoT devices 102 at site A/X,and may cache and/or forward the data 103 to the network node(s) 106 forsubsequent processing and activity state monitoring. Likewise, feedbacksignals that actualize the computed activity state via feedback devices112 may be routed through an actuator device or module 110B/110Y in siteA and/or site X respectively. This actuator may decode, parse, and/orperform other processing of the feedback signals such that the activitystate is properly rendered to the participant when forwarded to the oneor more feedback devices 112 in the feedback device set associated witha given site.

Turning to the network side operation, once the network node 106 hasreceived the data 103, the network node 106 may store the observed IoTdevice data 103 in a local memory (or in memory of another accessibledevice). Based on the data received from the IoT devices 102, dataprocessor and state engine implemented on network node 106 may process astate of the activity indicated by the data 103 and/or may predictevents that may occur at a future time. For instance, in the squashexample, an IoT camera, accelerometer, or smart squash racquet device102 may initially send data to the network node 106 that indicates aball or racquet is moving at a certain speed and direction, and fromthat data, the data processor/state engine 104 can predict that the ballis being struck (or will be struck) or is hitting a wall (or will hit awall) in the squash court (the wall contact being the present orpredicted event) at the sensor-observed time or particular estimatedtime in the future.

These events, along with observed parameter values indicatingenvironmental and time-specific activity conditions, can make up acurrent “state” of the activity. Such states may be defined by applyingthe observed data 103 to a preconfigured, later-obtained, static, ordynamic definitional instruction set, which is referred to herein asdata model 105. To calculate an activity state, the data processor/stateengine may query the data model 105 for information regarding how astate is to be defined and generated in view of any obtained set of IoTdevice data 103. Once the state has been defined, the dataprocessor/state engine 104 can further generate, based on the generatedactivity state and the composition of site-specific feedback device sets(e.g., 112X and 112Y for site X), a corresponding set of feedbacksignals that, when received by the feedback devices 112, actualize thestate of the activity for each participant via his or her availablefeedback devices 112. For instance, this may include delivering commandsto a speaker to emit a sound, a projector above a squash court to renderan image or hologram of a ball or other participant(s) in a particularlocation on the rendered court map, a handheld video game console tovibrate and display a state-specific 2-dimensional display screenrendering, or any other sensory feedback to the users. Furthermore, notonly are each of these feedback signals generated and rendered on aparticipant/device-specific granularity, but each of the unique feedbacksignal sets destined for each particular site can indicate a same (orsubstantially the same) rendering of a state, with any differences inthe actualized activity state being defined by the device-imposedlimitations of the feedback device set of each unique activity site.

This IoT device capture to state generation to tailored stateactualization process may be repeated at a particular frequency suchthat the state is rendered, at least from the participants' perspective,continuously and in substantially real time. However, this may notalways be the case, as limitations imposed by available network anddevice processing power and time-frequency resources may limit theperformance capability of the system. In such instances, the networknode(s) 106 (or any other controlling device, which may optionallyinclude any device or node included or not shown in FIG. 1) may throttlethe amount of data 103 reported to the network nodes 106, the frequencyof such reporting or capture (from a blanket rule to a device-specificpolicy change or even to an internal device setting alteration for asingle device), or the degree of processing power or medium/channelresources throttling utilized on the network and/or device side. Suchalterations may be implemented, for example, in an attempt to meet aQuality-of-Experience (QoE)/Quality of Service (QoS), throughput, or anyother performance-defining metric/criterion of the system as applied tothe ongoing participant activity.

As shown in FIG. 1, after determining any adapted IoT device settingsbased on the target optimization function, the network node 106 candeliver any adapted settings (or all settings, regardless of changes, insome examples) to appropriate IoT devices 102. In addition, network-sizeparameters, such as bandwidth allocation, data flow QoS, allocatedprocessing power, and the like can be implemented according to anyresulting adaptation. Thus, based on the actual and forecastedperformance requirements and resource constraints of the system, thenetwork nodes 106 or any other controlling device continuously optimizesthe resource allocation by allocating an appropriate quantity and volumeof underlying network resources (e.g., time and frequency resources,processing power, central processing units (CPUs)/graphics processingunits (GPUs)/cores, memory, disk, etc.) and by controlling thegranularity of sensor data 103 to be collected and uploaded by the IoTdevices 102 for subsequent state generation, rendering, and/orprediction.

As such, in sum, the example implementation of FIG. 1 illustratesexample devices and functional blocks of a proposed IoT system 100. Inthe lowest layer, at the edge of the IoT system, there are IoT devices102, such as sensors (cameras, accelerometers, etc.). On the networkside, network nodes 106 include memory and associated modules thatreceive and store data observed by one or more IoT devices 102(optionally via separate data collector modules 108), which may take theform of raw, unprocessed sensor-collected data. Within thecloud-resident node modules 106, there are data processor blocks thatare responsible for processing the uploaded data in order to distillinformation, in some cases events from the data 103 streams. The datamodel 105 defines the state of each activity/game session, which isoftentimes continuously refined be the data processor module(s) 104.When necessary, the data model 150 and/or data processor 104 generateand send feedback to the IoT system. This feedback may travel throughthe optional actuators 110 at the sites for processing and/or routingoperations before being forwarded to the sites such that theparticipants are able to consume sensory information via feedbackdevices 112 (e.g., via a projection on a squash wall, audio played byspeakers on the courts, and/or haptic feedback via vibration or otherforce feedback on racquets, etc.)

FIG. 2 illustrates an example method 200 performed by one or morenetwork nodes 106 for providing and maintaining an activity state thatis actualized for one or more activity participants. For instance, atblock 202, the network node 106 (e.g., via data collector module ordevice) can obtain data captured/observed by one or more IoT devices andassociated with an interactive activity with which differentparticipants interface at different sites.

In addition, at block 204, the network node 106 (e.g., via the dataprocessing/state engine module of FIG. 1) can determine a state of theactivity by applying the data to a data model.

In some instances, this may involve generating an effectively present orcurrent state of the activity, whereas it may additionally oralternatively involve generating a future state of the activity, and/orpredicting future events, with respect to individual participants.

Furthermore, method 200 may include, at block 206, providing one or morefeedback signals to feedback devices associated with one or more of thedifferent sites to actualize the determined state of the activity, atleast two of the different sites having disparate feedback device sets.By disparate, this means that, at minimum, at least two device setscomposed of the feedback devices available at two particular sitesassociated with two particular participants are non-identical.

Turning to FIG. 3, aspects discussed above in reference to FIGS. 1 and 2will be described in terms of the flow chart of method 300, which isrelated to the features of method 200. Returning to the squash example,for each game the network node 106 executes processes for maintaining astate of an activity. Again, such an activity state can include data,parameter values, static features, dynamic movements or changes relativeto prior or predicted states or events, results of history-informedprobabilistic event or characteristic forecasting, or any other featurethat defines an aspect of the activity at a particular time. This caninclude a temperature, real or virtually rendered/mapped object orparticipant position or movement (position delta), a wall ball impactlocation, a stroke, jump, injury, noise, air pressure, moisture level,score, participant number or characteristic, etc., and more generally,can be any feature whatsoever when the present aspects are implementedoutside of the squash context

Turning to method 300, FIG. 3 shows a process chain from data gatheringstep at block 302 to the feedback step at block 308 that determinesvideo and other feedback information that should be fed back to theparticipants at a given time to actuate a determined activity state. Inthe data collection step of block 302, data is collected from thesite-specific environments by IoT devices 102 (any type of sensor, e.g.,cameras). Next, at block 304, data is uploaded and processed in thecloud by the network node(s) 106, for instance, by decoding, parsing,and processing the data (including, in some instances, executingcustom-made event detection algorithms, e.g., object/participanttracking). In the third step of block 306, the computed parameters anddata associated with detected events or conditions are loaded into thedata model, and the current model state of the observed system isestablished. In the fourth illustrated step of block 308, based on thecurrent state defined by the data model and generated by data processingmodules, feedback is provided to feedback devices included in particularfeedback device sets associated with each participant (e.g., an imageprojected on a squash court wall is updated from a previously actualizedstate).

FIG. 4 illustrates additional details of an example network node 106 ofan IoT system according to one or more embodiments. The network node 106is configured, e.g., via functional means or units (also may be referredto as modules or components herein), to implement processing to performcertain aspects described above in reference to at least FIGS. 1-3. Thenetwork node 106 in some embodiments for example includes a dataobtaining means or unit 440 for obtaining IoT-device-reported data, adata processing and state management means or unit 460 (which maycorrespond to data processor/state engine 104 of FIG. 1) for processingthe obtained data to generate a state associated with an activity, and afeedback providing means or unit for sending site- andfeedback-device-set-specific feedback instructions/commands to one ormore activity sites (may correspond to data model 105 and/or dataprocessor/state engine 104 of FIG. 1). These and potentially otherfunctional means or units (not shown) together perform the aspects ofmethods 200 and 300 presented in FIG. 3 and/or further or alternativefeatures described in FIGS. 1-3 as being related to the network node106.

In at least some embodiments, the network node 106 comprises one or moreprocessing circuits 420 configured to implement processing of the method200 of FIG. 2, method 300 of FIG. 3, and certain associated processingof the features described in relation to FIG. 1, such as by implementingfunctional means or units above. In one embodiment, for example, theprocessing circuit(s) 420 implements functional means or units asrespective circuits. The circuits in this regard may comprise circuitsdedicated to performing certain functional processing and/or one or moremicroprocessors in conjunction with memory 430. In embodiments thatemploy memory 430, which may comprise one or several types of memorysuch as read-only memory (ROM), random-access memory, cache memory,flash memory devices, optical storage devices, etc., the memory 430stores program code that, when executed by the one or moremicroprocessors, causes the network node to carry out the techniquesdescribed herein.

In one or more embodiments, the network node 106 also comprises one ormore communication interfaces 410. The one or more communicationinterfaces 410 include various components (e.g., antennas 440) forsending and receiving data and control signals. More particularly, theinterface(s) 410 include a transmitter that is configured to use knownsignal processing techniques, typically according to one or morestandards, and is configured to condition a signal for transmission(e.g., over the air via one or more antennas 440). Similarly, theinterface(s) include a receiver that is configured to convert signalsreceived (e.g., via the antenna(s) 440) into digital samples forprocessing by the one or more processing circuits. In an aspect, thedata obtaining means or unit 440 and/or the feedback providing means orunit 480 may comprise or may be in communication with the transmitterand/or receiver. The transmitter and/or receiver may also include one ormore antennas 440.

FIG. 5 illustrates additional details of an example IoT device 102according to one or more embodiments. The IoT device 102 is configured,e.g., via functional means or units (also may be referred to as modulesor components herein), to implement processing to perform certainaspects described above in reference to FIGS. 1-3 herein. The IoT device102 in some embodiments for example includes a data transmission meansor unit 550 for transmitting captured data to a network node 105, and adata capture and setting means or unit for capturing data according toone or more settings, such as updating settings values according tocommands received from a network node 105. These and potentially otherfunctional means or units (not shown) together perform the IoT featuresdescribed in FIGS. 1-3 as being related to the IoT device 102.

In at least some embodiments, the IoT device 102 comprises one or moreprocessing circuits 520 configured to implement processing of the method200 of FIG. 2 and certain associated processing of the featuresdescribed in relation to IoT device 102 to FIGS. 1 and 3, such as byimplementing functional means or units above. In one embodiment, forexample, the processing circuit(s) 520 implements functional means orunits as respective circuits. The circuits in this regard may comprisecircuits dedicated to performing certain functional processing and/orone or more microprocessors in conjunction with memory 530. Inembodiments that employ memory 530, which may comprise one or severaltypes of memory such as read-only memory (ROM), random-access memory,cache memory, flash memory devices, optical storage devices, etc., thememory 530 stores program code that, when executed by the one or moremicroprocessors, causes the device to carry out the techniques describedherein.

In one or more embodiments, the IoT device 102 also comprises one ormore communication interfaces 510. The one or more communicationinterfaces 510 include various components (e.g., antennas 540) forsending and receiving data and control signals. More particularly, theinterface(s) 510 include a transmitter that is configured to use knownsignal processing techniques, typically according to one or morestandards, and is configured to condition a signal for transmission(e.g., over the air via one or more antennas 540). In an aspect, therevealing module or unit 550 may comprise or may be in communicationwith the transmitter. Similarly, the interface(s) include a receiverthat is configured to convert signals received (e.g., via the antenna(s)540) into digital samples for processing by the one or more processingcircuits. The transmitter and/or receiver may also include one or moreantennas 540.

Those skilled in the art will also appreciate that embodiments hereinfurther include corresponding computer programs. A computer programcomprises instructions which, when executed on at least one processor ofthe network node 106, feedback device 112, or IoT device 102 cause thesedevices to carry out any of the respective processing described above.Furthermore, the processing or functionality of network node 106 may beconsidered as being performed by a single instance or device or may bedivided across a plurality of instances of network node 106 that may bepresent in a given cloud network such that together the device instancesperform all disclosed functionality. In addition, network node 106 maybe any known type of device associated with a cloud network, radiocommunication network, or content delivery network, generally, that isknown to perform a given disclosed processes or functions thereof.Examples of such network nodes include eNBs, Mobility ManagementEntities (MMEs), gateways, servers, and the like.

Embodiments further include a carrier containing such a computerprogram. This carrier may comprise one of an electronic signal, opticalsignal, radio signal, or computer readable storage medium. A computerprogram in this regard may comprise one or more code modulescorresponding to the means or units described above.

A network node 106 herein is any type of network node (e.g., a basestation) capable of communicating with another node over radio signals.IoT device 102 is any type device capable of communicating with a radionetwork node 10 over radio signals, such as, but not limited to, adevice capable of performing autonomous wireless communication with oneor more other devices, including a machine-to-machine (M2M) device, amachine-type communications (MTC) device, a user equipment (UE) (howeverit should be noted that the UE does not necessarily have a “user” in thesense of an individual person owning and/or operating the device). AnIoT device may also be referred to as a radio device, a radiocommunication device, a wireless terminal, or simply a terminal—unlessthe context indicates otherwise, the use of any of these terms isintended to include device-to-device UEs or devices, machine-typedevices or devices capable of machine-to-machine communication, sensorsequipped with a wireless device, wireless-enabled table computers,mobile terminals, smart phones, laptop-embedded equipped (LEE),laptop-mounted equipment (LME), USB dongles, wireless customer-premisesequipment (CPE), etc. In the discussion herein, the termsmachine-to-machine (M2M) device, machine-type communication (MTC)device, wireless sensor, and sensor may also be used. It should beunderstood that these devices may be UEs, but are generally configuredto transmit and/or receive data without (or with minimal) direct humaninteraction.

In any scenario discussed above, the IoT device 102 herein may be, ormay be comprised in, a machine or device that performs monitoring ormeasurements, and transmits the results of such monitoring measurementsto another device or a network. Particular examples of such machines arepower meters, industrial machinery, or home or personal appliances, e.g.refrigerators, televisions, personal wearables such as watches etc. Inother scenarios, a wireless communication device as described herein maybe comprised in a vehicle and may perform monitoring and/or reporting ofthe vehicle's operational status or other functions associated with thevehicle.

In the above examples, the gestures may be recorded with sensors, e.g.,IMU (Inertial Measurement Unit) sensor-equipped squash racquets,wristbands, shoes, etc., while the movements of the players and of thesquash balls may be recorded by cameras and/or microphones mounted onthe squash court walls. Sensor (and audio and video) data may beprocessed in the cloud, and fed back to the (remote) players on eachcourt by a projector, mounted on, e.g., the ceiling and projectingimages and/or video to the (front) wall(s) and/or to the floor.Additionally or alternatively, a possible way of information feedback toplayers in a squash court could be implemented via haptic devicesinstalled in the squash racquets and/or wrist bands.

The inventive nature of the proposed system stems, at least in part,from the novel multi-purpose usage of cheap sensors mounted on a squashcourt and on sports equipment. The input and output interfaces may beimplemented by the aforementioned cheap gears, which may already benecessary and in place in order to enrich traditional sportingexperience. The computations related to the actual games may beperformed in the cloud, as opposed to local computation performed ingame consoles, and the connectivity of the sport spaces, already inplace, may be used to transfer the sensor data up to the cloud, and theuser interface and feedback back to the sport spaces.

The proposed system implements a single- or multi-player interactivevideo game that can be played by players on separate squash courtslocated in a distance from one another. The gestures of the players maybe recorded with sensors, e.g., IMU (Inertial Measurement Unit)sensor-equipped squash racquets, wristbands, shoes, etc., while themovements of the players and of the squash balls may be recorded bycameras and/or microphones mounted on the squash court walls. Sensor(and audio and video) data may be processed in the cloud, and fed backto the (remote) players on each court by a projector, mounted, forexample, on the ceiling and projecting images and/or video to the(front) wall(s) and/or to the floor. Additionally or alternatively, apossible way of information feedback to players in a squash court couldbe implemented via haptic devices installed in the squash racquetsand/or wrist bands.

The proposed system is effective partly because it can utilize cheapsensors mounted on a squash court and on sports equipment. The input andoutput interfaces may be implemented by the aforementioned cheap gears,already necessary and in place in order to enrich traditional sportingexperience. The computations related to the actual games may beperformed in the cloud, as opposed to local computation performed ingame consoles, and the connectivity of the sport spaces, already inplace, is used to transfer the sensor data up to the cloud, and the userinterface and feedback back to the sport spaces. Some embodiments arepresented herein that can be implemented in the proposed system withvarious level of complexity, i.e., number and diversity of sensors andactuators to be applied for the given use case. With these examples, itis intended to show that the input devices, the collected data streams,the data processing steps and the output devices of the multiplayer gamecan be different across the players, the smart sports equipment and thesmart sport spaces being only a subset of the potential interconnecteddevices.

One example embodiment relates to playing catch. Many, remotely situatedsquash courts can be used to play catch among players on all thosecourts. In order to do this, the necessary set of sensors containsdevices that can track the players on each court, and the necessary setof actuators contains devices that can distinguish the catcher andpinpoint the catcher's position on all courts at all times (every playerbeing on a common virtual field). A possible, sufficient set of devicescomprises of one ceiling-mounted camera on each court with a view on thewhole court surface, a lighting wrist band on each player that signalsred on the catcher, and a ceiling-mounted projector that projects a redspot to the respective position of the catcher on all courts.

Another example embodiment relates to playing a brick breaker gamebetween a squash court and a smartphone. The brick breaker game can beplayed among players that either play in one or more squash courts, oron their smartphones. A sufficient set of devices to support this is thefollowing: a back-wall-mounted camera on each squash court to detect thepositions of ball impacts on the front wall, motion sensors in eachsquash racquet in order to link ball impacts to players by detecting thestrokes prior to impacts, a projector on each court to project the brickbreaker game on the front wall, and a mobile application client at eachplayer that plays on their smartphone to detect the touches on thescreen and to render the brick breaker game on the screen. Note that inthis case the physical dimensions of the common brick wall are not thesame on every output visualization, i.e., front wall of a squash courtvs. screen size of a smartphone.

FIG. 6 shows an example of the process chain 600 from data gathering tothe feedback step that determine video and other information that mayhave to be fed back to the player(s) at a given time. In the first step,data is collected from the environment by any type of sensors, e.g.,cameras, then in the second step the data is uploaded and processed inthe cloud by custom-made event detection algorithms, e.g., balltracking. In the third step the computed parameters are loaded in thepredefined data model, and the current model state of the observedsystem is established. In the fourth step, based on the current stateand the data model, feedback is given to the player(s), e.g., the imageprojected on the wall is updated.

FIG. 7 shows the functional blocks of an example of the proposed system700. In the lowest layer, at the edge of the IoT system, there aresensors (cameras, accelerometers, etc.) and the Data collector moduleswith uplink capacity that is used for uploading the raw, unprocessedsensor-collected data to the cloud. Within the cloud there are Dataprocessor blocks that are responsible for processing the uploaded datain order to distill information, in most cases events from the datastreams. The Data model keeps track of each game session and it iscontinuously refined be the Data processor module(s). When necessary,the Data model sends feedback to the IoT system, i.e., to the Actuatorsat the sites so that those can visualize information on the wall, and/orplay audio in the courts, and/or give haptic feedback via racquets, etc.to the players. The block diagram above represents 2 sites, 2 and 1courts in each respectively, geared with 2, 2 and 1 cameras, 1, 0 and 1projectors, 0, 0 and 2 motion sensors and haptic devices mounted onracquets.

FIG. 8 shows an example of communication links between the elements ofthe system 800, comprising two sites in this example. Within the site,the sensors and the various feedback (visual, haptic, etc.) devices areconnected to the Data collector and to the Actuator elementsrespectively, over wireless communication technologies, or possibly overwired communication technologies. These two elements act as gatewaystowards the cloud over the Internet. Therefore the communication betweenthe Data collector, the Actuator and the Cloud is optionally performedover Internet Protocol (IP), probably via wired links, or possibly viawireless access.

Embodiments outlined herein present a novel and nonobvious combinationof sports equipment with video gaming via Internet of Things and Cloudcomputing technologies. Specifically, for squash, a video game ispresented that can be played by two (or more) players that hit the frontwall with a squash ball in remote squash courts simultaneously. Themultiplayer online games played with sports equipment (either gearedwith sensors and/or surrounded by tracking sensors mounted on the sportspace) can be cooperative and/or competitive. By emulating and/or 3Dprojecting the only ball players remotely “play with”, even regularsport matches can be played by players in a distance from each other.

Advantages of embodiments of the network node of an IoT system and ofthe method executed by a network node in an IoT system as describedherein include one or more of: extending the features of sport spaces,for example, squash courts, with multi-player video gaming, reusingsmart sport variable devices for video gaming, substituting home gameconsoles with cheaper input devices at public sport spaces, andconnecting videogame players with sports. The cloud-based solutionherein takes significantly more input than existing systems, and cantherefore produce fully interactive feedback environments for anactivity across disparate activity participation sites.

The present embodiments may, of course, be carried out in other waysthan those specifically set forth herein without departing fromessential characteristics of the invention. The present embodiments areto be considered in all respects as illustrative and not restrictive,and all changes coming within the meaning and equivalency range of theappended claims are intended to be embraced therein.

1-20. (canceled)
 21. A method executed by a network node in anInternet-of-Things (IoT) system, the method comprising: obtaining datacaptured by one or more IoT devices and associated with an interactiveactivity with which different participants interface at different sites;determining a state of the activity by applying the data to a datamodel; and providing one or more feedback signals to feedback devicesassociated with one or more of the different sites to actualize thedetermined state of the activity, at least two of the different siteshaving disparate feedback device sets.
 22. The method of claim 21,wherein the providing the one or more feedback signals comprisesactualizing the determined state of the activity by applying differentforms of sensory feedback to the different participants via a feedbackdevice set associated with a site of each of the different participants.23. The method of claim 22, wherein the different forms of sensoryfeedback are applied based on the state of the activity shared by eachof the different sites.
 24. The method of claim 21, wherein the statecomprises a state of the activity at a time at which the data iscaptured by the one or more IoT devices, a state at a current time, apredicted state of the activity at a future time, or a combinationthereof.
 25. The method of claim 21, wherein the determining the statecomprises identifying a location of one or more of the differentparticipants in a common virtual activity space.
 26. The method of claim21, wherein at least two of the different sites have disparate IoTdevice sets.
 27. The method of claim 21, wherein at least one of theparticipants interfaces with the interactive activity virtually via aninput-output device.
 28. The method of claim 21, wherein at least one ofthe participants interfaces with the interactive activity in afeedback-device-rendered variation of a traditional real-life form ofthe activity.
 29. The method of claim 21, wherein the activity comprisesa competition between or among a plurality of the different participantsor a competition between a single participant against a benchmark orprevious score.
 30. The method of claim 21, wherein the disparatefeedback device sets comprise at least two sets of feedback deviceswhose included device types are non-identical.
 31. The method of claim21, wherein the network node comprises a plurality of network devicesworking in collaboration.
 32. The method of claim 21, wherein the IoTsystem comprises a cloud computing system.
 33. A network node in anInternet-of-Things (IoT) system, the network node comprising: processingcircuitry; memory containing instructions executable by the processingcircuitry whereby the network node is operative to: obtain data capturedby one or more IoT devices and associated with an interactive activitywith which different participants interface at different sites;determine a state of the activity by applying the data to a data model;and provide one or more feedback signals to feedback devices associatedwith one or more of the different sites to actualize the determinedstate of the activity, at least two of the different sites havingdisparate feedback device sets.
 34. A non-transitory computer readablerecording medium storing a computer program product for controlling anetwork node in an Internet-of-Things (IoT) system, the computer programproduct comprising software instructions which, when run on processingcircuitry of the network node, causes the network node to: obtain datacaptured by one or more IoT devices and associated with an interactiveactivity with which different participants interface at different sites;determine a state of the activity by applying the data to a data model;and provide one or more feedback signals to feedback devices associatedwith one or more of the different sites to actualize the determinedstate of the activity, at least two of the different sites havingdisparate feedback device sets.